What Determines Human Certainty?

Previous work on concept learning has focused on how concepts are acquired without addressing metacognitive aspects of this process. An important part of concept learning from a learner’s perspective is subjectively knowing when a new concept has been effectively learned. Here, we investigate learners’ certainty in a classic Boolean concept-learning task. We collected certainty judgements during the concept-learning task from 552 participants on Amazon Mechanical Turk. We compare different models of certainty in order to determine exactly what learners’ subjective certainty judgments encode. Our results suggest that learners’ certainty is best explained by local accuracy rather than plausible alternatives such as total entropy or the maximum a posteriori hypothesis of an idealized Bayesian learner. This result suggests that certainty predominately reflects learners’ performance and feedback, rather than any metacognition about the inferential task they are solving.